In an effort to overcome the shortcomings of conventional neural networks, holographic neural networks have been developed. Holographic neural networks include two vectors and one matrix, all of complex numbers. The two vectors include the S or stimulus vector (length n) and the R or response vector (length m). The matrix is the X matrix (size n.times.m) that describes a mapping from S to R. These networks offer several advantages over conventional neural networks including that they are simpler to train and discern. In addition, while conventional neural networks are usually structured to approximate digital quantities, holographic neural networks are well suited to representing continuously valued mappings.
Notwithstanding these comparative benefits, first order holographic neural networks are limited in the complexity of the mapping that can be encoded. This is because (1) the X matrix has a finite capacity to contain encodings and (2) the encoding is unable to represent complex interrelationships between discrete components in the stimulus vector.
These disadvantageous aspects are addressed, however, through the provision of higher order terms in the stimulus vector (where a higher order term is defined as a product of two or more initial input first-order terms). The generation of higher order terms allows a large increase in the size and therefore encoding capacity of the network matrix. For example, from 10 distinct first-order terms, 55 distinct second-order terms and 220 distinct third-order terms can be generated. Higher order terms also represent interactions between discrete first-order components of the stimulus vector, allowing the holographic neural network to represent such interactions in a manner similar to that of conventional neural networks. Finally, higher order terms tend to be more evenly distributed through the phase space than first-order terms and this also increases encoding capacity.
With the availability of higher order terms, the design of a holographic neural network is directed toward the selection of which higher order terms to include in the stimulus vector. This choice can be extremely difficult and is a primary design challenge in the creation of useful holographic neural networks.